Effects of the rate of formant-frequency variation on the grouping of formants in speech perception

How speech is separated perceptually from other speech remains poorly understood. Recent research suggests that the ability of an extraneous formant to impair intelligibility depends on the modulation of its frequency, but not its amplitude, contour. This study further examined the effect of formant-frequency variation on intelligibility by manipulating the rate of formant-frequency change. Target sentences were synthetic three-formant (F1?+?F2?+?F3) analogues of natural utterances. Perceptual organization was probed by presenting stimuli dichotically (F1?+?F2C?+?F3C; F2?+?F3), where F2C?+?F3C constitute a competitor for F2 and F3 that listeners must reject to optimize recognition. Competitors were derived using formant-frequency contours extracted from extended passages spoken by the same talker and processed to alter the rate of formant-frequency variation, such that rate scale factors relative to the target sentences were 0, 0.25, 0.5, 1, 2, and 4 (0?=?constant frequencies). Competitor amplitude contours were either constant, or time-reversed and rate-adjusted in parallel with the frequency contour. Adding a competitor typically reduced intelligibility; this reduction increased with competitor rate until the rate was at least twice that of the target sentences. Similarity in the results for the two amplitude conditions confirmed that formant amplitude contours do not influence across-formant grouping. The findings indicate that competitor efficacy is not tuned to the rate of the target sentences; most probably, it depends primarily on the overall rate of frequency variation in the competitor formants. This suggests that, when segregating the speech of concurrent talkers, differences in speech rate may not be a significant cue for across-frequency grouping of formants.

Wideband inhibition modulates the effect of onset asynchrony as a grouping cue

Onset asynchrony is arguably the most powerful grouping cue for the separation of temporally overlapping sounds (see Bregman 1990). A component that begins only 30–50 ms before the others makes a greatly reduced contribution to the timbre of a complex tone, or to the phonetic quality of a vowel (e.g. Darwin 1984). This effect of onset asynchrony does not necessarily imply a cognitive grouping process; instead it may result from peripheral adaptation in the response to the leading component in the few tens of milliseconds before the other components begin (e.g., Westerman and Smith 1984). However, two findings suggest that the effect of onset asynchrony cannot be explained entirely by peripheral adaptation. First, though the effect is smaller, the contribution of a component to the phonetic quality of a short-duration vowel is reduced when it ends after the other components (Darwin and Sutherland 1984; Roberts and Moore 1991).

Contralateral influences of wideband inhibition on the effect of onset asynchrony as a cue for auditory grouping

Onset asynchrony is an important cue for segregating sound mixtures. A harmonic of a vowel that begins before the other components contributes less to vowel quality. This asynchrony effect can be partly reversed by accompanying the leading portion of the harmonic with an octave-higher captor tone. The original interpretation was that the captor and leading portion formed a perceptual group, but it has recently been shown that the captor effect depends on neither a common onset time nor harmonic relations with the leading portion. Instead, it has been proposed that the captor effect depends on wideband inhibition in the central auditory system. Physiological evidence suggests that such inhibition occurs both within and across ears. Experiment 1 compared the efficacy of a pure-tone captor presented in the same or opposite ear to the vowel and leading harmonic. Contralateral presentation was at least as effective as ipsilateral presentation. Experiment 2 used multicomponent captors in a more comprehensive evaluation of harmonic influences on captor efficacy. Three captors with different fundamental frequencies were used, one of which formed a consecutive harmonic series with the leading harmonic. All captors were equally effective, irrespective of the harmonic relationship. These findings support and refine the inhibitory account. © 2007 Acoustical Society of America.

Grouping and the pitch of a mistuned fundamental component:effects of applying simultaneous multiple mistunings to the other harmonics

Mistuning a harmonic produces an exaggerated change in its pitch. This occurs because the component becomes inconsistent with the regular pattern that causes the other harmonics (constituting the spectral frame) to integrate perceptually. These pitch shifts were measured when the fundamental (F0) component of a complex tone (nominal F0 frequency = 200 Hz) was mistuned by +8% and -8%. The pitch-shift gradient was defined as the difference between these values and its magnitude was used as a measure of frame integration. An independent and random perturbation (spectral jitter) was applied simultaneously to most or all of the frame components. The gradient magnitude declined gradually as the degree of jitter increased from 0% to ±40% of F0. The component adjacent to the mistuned target made the largest contribution to the gradient, but more distant components also contributed. The stimuli were passed through an auditory model, and the exponential height of the F0-period peak in the averaged summary autocorrelation function correlated well with the gradient magnitude. The fit improved when the weighting on more distant channels was attenuated by a factor of three per octave. The results are consistent with a grouping mechanism that computes a weighted average of periodicity strength across several components. © 2006 Elsevier B.V. All rights reserved.

Spectral pattern, grouping, and the pitches of complex tones and their components

Harmonically related components are typically heard as a unified entity with a rich timbre and a pitch corresponding to the fundamental frequency. Mistuning a component generally has four consequences: (i) the global pitch of the complex shifts in the same direction as the mistuning; (ii) the component makes a reduced contribution to global pitch; (iii) the component is heard out as a separate sound with a pure timbre; (iv) its pitch differs from that of a pure tone of equal frequency in a small but systematic way. Local interactions between neighbouring components cannot explain these effects; instead they are usually explained in terms of the global operation of a single harmonic-template mechanism. However, several observations indicate that separate mechanisms govern the selection of spectral components for perceptual fusion and for the computation of global pitch. First, an increase in mistuning causes a harmonic to be heard out before it begins to be excluded from the computation of global pitch. Second, a single even harmonic added to an odd-harmonic complex is typically more salient than its odd neighbours. Third, the mistuning of a component in frequency-shifted stimuli, or stimuli with a moderate spectral stretch, results in changes in salience and component pitch like those seen for harmonic stimuli. Fourth, the global pitch of frequency-shifted stimuli is predicted well by the weighted fit of a harmonic template, but, with the exception of the lowest component, the fusion of individual partials for shifted stimuli is best predicted by the common pattern of spectral spacing. Fifth, our sensitivity to spectral pattern is surprisingly resistant to random variations in component spacing induced by applying mistunings to several harmonics at once. These findings are evaluated in the context of an autocorrelogram model of the proposed pitch/grouping dissociation. © S. Hirzel Verlag · EAA.

Asynchrony and the grouping of vowel components:captor tones revisited

Asynchrony is an important grouping cue for separating sound mixtures. A harmonic incremented in level makes a reduced contribution to vowel timbre when it begins before the other components. This contribution can be partly restored by adding a captor tone in synchrony with, and one octave above, the leading portion of the incremented harmonic [Darwin and Sutherland, Q. J. Exp. Psychol. A 36, 193-208 (1984)]. The captor is too remote to evoke adaptation in peripheral channels tuned to the incremented harmonic, and so the restoration effect is usually attributed to the grouping of the leading portion with the captor. However, results are presented that contradict this interpretation. Captor efficacy does not depend on a common onset, or harmonic relations, with the leading component. Rather, captor efficacy is influenced by frequency separation, and extends to about 1.5 oct above the leading component. Below this cutoff, the captor effect is equivalent to attenuating the leading portion of the incremented harmonic by about 6 dB. These results indicate that high-level grouping does not govern the captor effect. Instead, it is proposed that the partial restoration of the contribution of an asynchronous component to vowel timbre depends on broadband inhibition within the central auditory system. © 2006 Acoustical Society of America.

Effects of differences in fundamental frequency on across-formant grouping in speech perception

In an isolated syllable, a formant will tend to be segregated perceptually if its fundamental frequency (F0) differs from that of the other formants. This study explored whether similar results are found for sentences, and specifically whether differences in F0 (?F0) also influence across-formant grouping in circumstances where the exclusion or inclusion of the manipulated formant critically determines speech intelligibility. Three-formant (F1 + F2 + F3) analogues of almost continuously voiced natural sentences were synthesized using a monotonous glottal source (F0 = 150 Hz). Perceptual organization was probed by presenting stimuli dichotically (F1 + F2C + F3; F2), where F2C is a competitor for F2 that listeners must resist to optimize recognition. Competitors were created using time-reversed frequency and amplitude contours of F2, and F0 was manipulated (?F0 = ±8, ±2, or 0 semitones relative to the other formants). Adding F2C typically reduced intelligibility, and this reduction was greatest when ?F0 = 0. There was an additional effect of absolute F0 for F2C, such that competitor efficacy was greater for higher F0s. However, competitor efficacy was not due to energetic masking of F3 by F2C. The results are consistent with the proposal that a grouping “primitive” based on common F0 influences the fusion and segregation of concurrent formants in sentence perception.

Simultaneous grouping in cochlear implant listeners:can abrupt changes in level be used to segregate components from a complex tone?

A sudden increase in the amplitude of a component often causes its segregation from a complex tone, and shorter rise times enhance this effect. We explored whether this also occurs in implant listeners (n?=?8). Condition 1 used a 3.5-s “complex tone” comprising concurrent stimulation on five electrodes distributed across the array of the Nucleus CI24 implant. For each listener, the baseline stimulus level on each electrode was set at 50% of the dynamic range (DR). Two 1-s increments of 12.5%, 25%, or 50% DR were introduced in succession on adjacent electrodes within the “inner” three of those activated. Both increments had rise and fall times of 30 and 970 ms or vice versa. Listeners reported which increment was higher in pitch. Some listeners performed above chance for all increment sizes, but only for 50% increments did all listeners perform above chance. No significant effect of rise time was found. Condition 2 replaced amplitude increments with decrements. Only three listeners performed above chance even for 50% decrements. One exceptional listener performed well for 50% decrements with fall and rise times of 970 and 30 ms but around chance for fall and rise times of 30 and 970 ms, indicating successful discrimination based on a sudden rise back to baseline stimulation. Overall, the results suggest that implant listeners can use amplitude changes against a constant background to pick out components from a complex, but generally these must be large compared with those required in normal hearing. For increments, performance depended mainly on above-baseline stimulation of the target electrodes, not rise time. With one exception, performance for decrements was typically very poor.

Sequential grouping of pure-tone percepts evoked by the segregation of components from a complex tone

A sudden change applied to a single component can cause its segregation from an ongoing complex tone as a pure-tone-like percept. Three experiments examined whether such pure-tone-like percepts are organized into streams by extending the research of Bregman and Rudnicky (1975). Those authors found that listeners struggled to identify the presentation order of 2 pure-tone targets of different frequency when they were flanked by 2 lower frequency “distractors.” Adding a series of matched-frequency “captor” tones, however, improved performance by pulling the distractors into a separate stream from the targets. In the current study, sequences of discrete pure tones were substituted by sequences of brief changes applied to an otherwise constant 1.2-s complex tone. Pure-tone-like percepts were evoked by applying 6-dB increments to individual components of a complex comprising harmonics 1–7 of 300 Hz (Experiment 1) or 0.5-ms changes in interaural time difference to individual components of a log-spaced complex (range 160–905 Hz; Experiment 2). Results were consistent with the earlier study, providing clear evidence that pure-tone-like percepts are organized into streams. Experiment 3 adapted Experiment 1 by presenting a global amplitude increment either synchronous with, or just after, the last captor prior to the 1st distractor. In the former case, for which there was no pure-tone-like percept corresponding to that captor, the captor sequence did not aid performance to the same extent as previously. It is concluded that this change to the captor-tone stream partially resets the stream-formation process, and so the distractors and targets became likely to integrate once more. (PsycINFO Database Record (c) 2011 APA, all rights reserved)

The influence of adaptation and inhibition on the effects of onset asynchrony on auditory grouping

Onset asynchrony is an important cue for auditory scene analysis. For example, a harmonic of a vowel that begins before the other components contributes less to the perceived phonetic quality. This effect was thought primarily to involve high-level grouping processes, because the contribution can be partly restored by accompanying the leading portion of the harmonic (precursor) with a synchronous captor tone an octave higher, and hence too remote to influence adaptation of the auditory-nerve response to that harmonic. However, recent work suggests that this restoration effect arises instead from inhibitory interactions relatively early in central auditory processing. The experiments reported here have reevaluated the role of adaptation in grouping by onset asynchrony and explored further the inhibitory account of the restoration effect. Varying the frequency of the precursor in the range ± 10% relative to the vowel harmonic (Experiment 1), or introducing a silent interval from 0 to 320 ms between the precursor and the vowel (Experiment 2), both produce effects on vowel quality consistent with those predicted from peripheral adaptation or recovery from it. However, there were some listeners for whom even the smallest gap largely eliminated the effect of the precursor. Consistent with the inhibitory account of the restoration effect, a contralateral pure tone whose frequency is close to that of the precursor is highly effective at restoring the contribution of the asynchronous harmonic (Experiment 3). When the frequencies match, lateralization cues arising from binaural fusion of the precursor and contralateral tone may also contribute to this restoration. (PsycINFO Database Record (c) 2012 APA, all rights reserved)

Axis-alignment affects perceptual grouping: evidence from simultanagnosia

We examined the effect of grouping by the alignment of implicit axes on the perception of multiple shapes, using a patient (GK) who shows simultanagnosia as part of Blint's syndrome. Five experiments demonstrated that: (1) GK was better able to judge the orientation of a global configuration if the constituent local shapes were aligned with their major axes than if they were aligned with their edges; (2) this axis information was used implicitly, since GK was unable to discriminate between configurations of axis-aligned and edge-aligned shapes; (3) GK's sensitivity to axis-alignment persisted even when the orientations of local shapes were kept constant, indicating some form of cooperative effect between the local elements; (4) axis-alignment of shapes also facilitated his ability to discriminate single-item from multi-item configurations; (5) the effect of axis-alignment could be attributed, at least partially, to the degree to which there was matching between the orientations of local shapes and the global configuration. Taken together, the results suggest that axis-based grouping can support the selection of multiple objects.

Axis-based grouping reduces visual extinction

We examined the effects on extinction of grouping by collinearity of edges and grouping by alignment of internal axes of shapes, in a patient (GK) with simultanagnosia following bilateral parietal brain damage. GK’s visual extinction was reduced when items (equilateral triangles and angles) could be grouped by base alignment (i.e., collinearity) or by axis alignment, relative to a condition in which items were ungrouped. These grouping effects disappeared when inter-item spacing was increased, though factors such as display symmetry remained constant. Overall, the results suggest that, under some conditions, grouping by alignment of axes of symmetry can have an equal beneficial effect on visual extinction as edge-based grouping; thus, in the extinguished field, there is derivation of axis-based representations from the contours present.

Neural networks for perpetual grouping

A number of researchers have investigated the application of neural networks to visual recognition, with much of the emphasis placed on exploiting the network's ability to generalise. However, despite the benefits of such an approach it is not at all obvious how networks can be developed which are capable of recognising objects subject to changes in rotation, translation and viewpoint. In this study, we suggest that a possible solution to this problem can be found by studying aspects of visual psychology and in particular, perceptual organisation. For example, it appears that grouping together lines based upon perceptually significant features can facilitate viewpoint independent recognition. The work presented here identifies simple grouping measures based on parallelism and connectivity and shows how it is possible to train multi-layer perceptrons (MLPs) to detect and determine the perceptual significance of any group presented. In this way, it is shown how MLPs which are trained via backpropagation to perform individual grouping tasks, can be brought together into a novel, large scale network capable of determining the perceptual significance of the whole input pattern. Finally the applicability of such significance values for recognition is investigated and results indicate that both the NILP and the Kohonen Feature Map can be trained to recognise simple shapes described in terms of perceptual significances. This study has also provided an opportunity to investigate aspects of the backpropagation algorithm, particularly the ability to generalise. In this study we report the results of various generalisation tests. In applying the backpropagation algorithm to certain problems, we found that there was a deficiency in performance with the standard learning algorithm. An improvement in performance could however, be obtained when suitable modifications were made to the algorithm. The modifications and consequent results are reported here.

Auditory grouping in cochlear implant listeners

This thesis describes a series of experiments investigating both sequential and concurrent auditory grouping in implant listeners. Some grouping cues used by normal-hearing listeners should also be available to implant listeners, while others (e.g. fundamental frequency) are unlikely to be useful. As poor spectral resolution may also limit implant listeners’ performance, the spread of excitation in the cochlea was assessed using Neural Response Telemetry (NRT) and the results were related to those of the perceptual tasks. Experiment 1 evaluated sequential segregation of alternating tone sequences; no effect of rate or evidence of perceptual ambiguity was found, suggesting that automatic stream segregation had not occurred. Experiment 2 was an electrode pitch-ranking task; some relationship was found between pitch-ranking judgements (especially confidence scores) and reported segregation. Experiment 3 used a temporal discrimination task; this also failed to provide evidence of automatic stream segregation, because no interaction was found between the effects of sequence length and electrode separation. Experiment 4 explored schema-based grouping using interleaved melody discrimination; listeners were not able to segregate targets and distractors based on pitch differences, unless accompanied by substantial level differences. Experiment 5 evaluated concurrent segregation in a task requiring the detection of level changes in individual components of a complex tone. Generally, large changes were needed and abrupt changes were no easier to detect than gradual ones. In experiment 6, NRT testing confirmed substantially overlapping simulation by intracochlear electrodes. Overall, little or no evidence of auditory grouping by implant listeners was found.

Formant-frequency variation and its effects on across-formant grouping in speech perception

How speech is separated perceptually from other speech remains poorly understood. In a series of experiments, perceptual organisation was probed by presenting three-formant (F1+F2+F3) analogues of target sentences dichotically, together with a competitor for F2 (F2C), or for F2+F3, which listeners must reject to optimise recognition. To control for energetic masking, the competitor was always presented in the opposite ear to the corresponding target formant(s). Sine-wave speech was used initially, and different versions of F2C were derived from F2 using separate manipulations of its amplitude and frequency contours. F2Cs with time-varying frequency contours were highly effective competitors, whatever their amplitude characteristics, whereas constant-frequency F2Cs were ineffective. Subsequent studies used synthetic-formant speech to explore the effects of manipulating the rate and depth of formant-frequency change in the competitor. Competitor efficacy was not tuned to the rate of formant-frequency variation in the target sentences; rather, the reduction in intelligibility increased with competitor rate relative to the rate for the target sentences. Therefore, differences in speech rate may not be a useful cue for separating the speech of concurrent talkers. Effects of competitors whose depth of formant-frequency variation was scaled by a range of factors were explored using competitors derived either by inverting the frequency contour of F2 about its geometric mean (plausibly speech-like pattern) or by using a regular and arbitrary frequency contour (triangle wave, not plausibly speech-like) matched to the average rate and depth of variation for the inverted F2C. Competitor efficacy depended on the overall depth of frequency variation, not depth relative to that for the other formants. Furthermore, the triangle-wave competitors were as effective as their more speech-like counterparts. Overall, the results suggest that formant-frequency variation is critical for the across-frequency grouping of formants but that this grouping does not depend on speech-specific constraints.